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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33), 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )S Responsive to communication(s) filed on 19 March 2004 . 
2a)Q This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except Tor formal matters, prosecution as to the merits is " 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-32 is/are pending in the application. 

4a) Of the above claim(s) 6-1 6 and 22-32 is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) £3 Claim(s) 1-5 and 17-21 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)Q accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
aM AH b)D Some * c)D None of: 

1 Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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1. Applicant's election with traverse of species I (claims 1-5 and 17-21) in Paper No. 
9 is acknowledged. The traversal is on the ground(s) that a search and examination of 
the entire application would not place a serious burden on the Examiner, whereas it 
would be a serious burden on Applicants. This is not found persuasive because the 
species are distinct and independent and do not relate to a single inventive concept as 
evidenced by the disclosure of each species in a separate patent application priority 
document. The four species are disclosed separately in four Japanese Patent 
Applications. The subject matter in the claims of the species lack unity of invention and 
the species are not obvious variants. Should applicant traverse on the ground that the 
species are not patentably distinct, applicant should submit evidence or identify such 
evidence now of record showing the species to be obvious variants or clearly admit on 
the record that this is the case. In either instance, if the examiner finds one of the 
inventions unpatentable over the prior art, the evidence or admission may be used in a 
rejection under 35 U.S.C. 103(a) of the other invention. 

The requirement is still deemed proper and is therefore made FINAL. 
Claim Rejections - 35 USC §112 

2. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

3. Claims 1-5, 17-21 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 
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4. Claim 1 recites the limitation "the individual images" in line 9. There is 
insufficient antecedent basis for this limitation in the claim. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 1, 4/1, 17, 20/17 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Onoguchi Kazunori (Japanese Patent Publication 
11-102440) "referred to as Onoguchi'440" and Onoguchi (U.S. Patent 5,694,483) 
"referred to as Onoguchi 1 483". Note, (U.S. Patent 6,205,242 is used as an Official 
translation of the subject matter disclosed in Japanese Patent Publication 11-102440). 

As to claim 1,Onogouchi'440 discloses an obstacle detection system comprising: 

a plurality of TV cameras for inputting multiple images (Fig. 1, TV cameras 1-N); 

an image storage unit for storing a plurality of images inputted from said TV 
cameras (Fig. 1, image memory 1-N); 

a parameter computation unit for determining a relation to hold between the 
projected positions of an arbitrary point on said plane upon the individual images, from 
the lines extracted by said feature extraction unit [each image is mapped onto a 
predetermined plane in a 3D space and a plurality of positions of sample points 
(parameters) are measured and the plane equation including the sample points (a 
relation to hold between the projected positions of an arbitrary point on said plane upon 
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the individual images) is determined . In this case a cross point area including a marked 
crosswalk (lines exiting in a plane) being monitored, a plurality of sample points are set 
on the road and positions of the sample points are determined (a relation to hold 
between the projected positions of an arbitrary point on said plane upon the individual 
images is determined from the marked crosswalk (lines)) (col. 4, lines 1-28 and Fig. 3)]; 
and 

a detection unit for detecting a region absent from said plane, by using the 
relation computed by said parameter computation unit [two projection data of same 
position on the plane P are compared and extracts an area R consisting of positions, 
the two projection data being different (col. 5, lines 39-42). More specifically, as shown 
in Fig. 8, for a point S' on the predetermined plane P, a projection point from TV camera 
1 is the same as a projection point from camera 2, in this case the difference in intensity 
Dif (i, j) is zero (i.e., the detected region is absent from the plane). On the other hand, 
for a head point S of the pedestrian (obstacle), the projection point Vi from TV camera 1 
is different from the projection point Ui from camera 2. In this case the difference in 
intensity Dif (i, j) is not zero it is of value "1" (col. 5, line 54- col. 6, Iine48, Fig. 8)]. 

Onoguchi'440, clearly determines the positions of a plurality of sample points set 
on a cross point area on a road including a marked crosswalk (marked crosswalk is 
lines existing on the road) (col. 4, lines 5-13), and it is clear that in order to determine 
the plurality of sample points set on the marked crosswalk (lines) of the road, the 
marked crosswalk has to be detected. Onoguchi'440 does not explicitly disclose 
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a feature extraction unit for extracting lines existing on a plane in a three- 
dimensional space, from the images. 

Onoguchi'483 discloses extracting a particular area from a road using stereo 
images (col. 3, lines 7-13). Extracting two pairs of white lines corresponding to edges of 
a road from a left image and a right image. Next, the stereo computation section 
extracts sample points from the two pairs of edges on each image and calculates the 
positions of the sample points in 3D space. Then, a plane calculation section calculates 
the equation of a plane according to the positions of the sample points (a relation to 
hold between the projected positions of an arbitrary point on said plane upon the 
individual images). A projection section, project the positions of each pixel of the left 
image and the right image onto a plane corresponding to the equation (col. 3, line54-col. 
4, Iine10). One of ordinary skill in the art would recognize from Onoguchi'483's teaching 
that in order to calculate or determine the plurality of sample points set on the marked 
crosswalk (lines) of the road disclosed in Onoguchi'440, the two pairs of white lines 
corresponding to edges of the marked crosswalk set on the road have to be first 
extracted and next extracts sample points from the two pairs of edges on each image 
and calculates the positions of the sample points in 3D space as taught by 
Onoguchi'483. It would have been obvious to one having ordinary skill in the art at the 
time the invention was made to use Onoguchi'483's teachings to modify Onoguchi'440's 
system by extracting lines existing on a plane in a three-dimensional space, from the 
images in order to accurately and correctly extract road areas from images without 
mistakenly extract other objects such as a building by the side of the road. 
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As to claim 4/1 , Onoguchi'483 further discloses, wherein said feature extraction 
unit extracts a plurality of lines, as existing on the plane in the three-dimensional space 
and parallel to each other in the three-dimensional space, from the images [Fig. 10, 
shows the two lines of the road are spaced a distance W and run parallel to each other 
in the 3D space of plane P], and determines the vanishing points of said lines [Fig. 3, 
shows the cross point (vanishing point) of the lines is determined]. 

As to claim 17 refer to claim 1 rejection. 

As to claim 20/17 refer to claim 4/1 rejection. 
7. Claims 2, 3/1, 3/2, 4/2, 4/3 and 18, 19/17, 19/18, 20/18, 20/19 are rejected under 
35 U.S.C. 103(a) as being unpatentable over the combination of Onoguchi Kazunori 
(Japanese Patent Publication 1 1-102440) "referred to as Onoguchi'440" and Onoguchi 
(U.S. Patent 5,694,483) "referred to as Onoguchi 1 483" as applied to claims 1 and 17 
above and further in view of Applicant's Admitted Prior Art (page 3, line 4-page 4, 
Iine18). 

As to claim 2, neither Onoguchi'440 nor Onoguchi'483 discloses, wherein said 
TV cameras are unknown on their relative positions and orientations and on their focal 
lengths and principal points. 

Applicant's admitted prior art discloses that in an ordinary stereovision system it 
is necessary to determine in advance the spacing and directions of the plurality of 
cameras and the focal lengths and the principal points of the camera lenses. This work 
for determining these parameters is called calibration. However calibration require a 
long time and many works. If the ground region and the obstacle region on the images 
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are sufficiently separated, the calibrations are not necessarily required. If the projected 
points of a point of the ground plane on the left and right images are designated (UI,VI) 
and (Ur,Vr), equation 1 is a relation to hold. Parameters h depend upon the locations 
and positions of the individual cameras with respect to the ground plane and upon the 
focal lengths and image origins of the lenses of the individual cameras. Parameters h 
are predetermined from the projected points of the ground plane on the left and right 
images (page3, line 4-page 4, line 18), i.e., by predetermining parameters h, no 
calibration is required for the stereo vision camera system and the spacing and 
directions of the plurality of cameras and the focal lengths and the principal points of the 
camera lenses are not required to be determined or known in advance. It would have 
been obvious to one having ordinary skill in the art at the time the invention was made 
to use Applicant's admitted prior art teachings to modify the combined system of 
Onoguchi'440 and Onoguchi'483 by predetermining parameters h in advance to avoid 
calibrating the TV cameras (i.e. not determining in advance the spacing and directions 
of the plurality of cameras and the focal lengths and the principal points of the camera 
lenses) in order to avoid the calibration operations of the cameras that require a long 
time and many work. 

As to claim 3/1 or 3/2, Applicant's admitted prior art further discloses, wherein the 
relation to hold between the projected points of an arbitrary point on the plane in the 
three-dimensional space upon the individual images is expressed by a two-dimensional 
affine transformation thereby to determine the affine transformation parameters 
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[equation 1 the relation to hold expressed in a two dimensional affine transformation, 
and parameters h are affine transformation parameters]. 

As to claim 18 refer to claim 2 rejection. 

As to claim 19/17 or 19/18 refer to claim 3/1 or 3/2 rejections. 

As to claims 4/2 or 4/3, Onoguchi'483 further discloses, wherein said feature 
extraction unit extracts a plurality of lines, as existing on the plane in the three- 
dimensional space and parallel to each other in the three-dimensional space, from the 
images [Fig. 10, shows the two lines of the road are spaced a distance W and run 
parallel to each other in the 3D space of plane P], and determines the vanishing points 
of said lines [Fig. 3, shows the cross point (vanishing point) of the lines is determined]. 

As to claims 20/1 8 or 20/1 9 refer to claim 4/2or 4/3 rejections. 
8. Claims 5/1 and 21/17 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over the combination of Onoguchi Kazunori (Japanese Patent Publication 1 1-102440) 
"referred to as Onoguchi'440" and Onoguchi (U.S. Patent 5,694,483) "referred to as 
Onoguchi' 483" as applied to claims 1 and 17 above and further in view of K. Storjohann 
et al., "Visual Obstacle Detection For Automatically Guided Vehicles", IEEE 
International Conference on Robotics and Automation, p761-766, 1990. 

As to claim 5/1 , Onoguchi'483 further discloses, wherein said feature extraction 
unit extracts a plurality of lines, as existing on the plane in the three-dimensional space 
and parallel to each other in the three-dimensional space, from the images [Fig. 10, 
shows the two lines of the road are spaced a distance W and run parallel to each other 
in the 3D space of plane P], and the vanishing points of said lines [Fig. 3, shows the 
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cross point (vanishing point) of the lines is determined]. Neither Onoguchi'440 nor 
Onoguchi' 483 

determines the inclinations of said lines on the images. 

Storjohann discloses an inverse perspective mapping for detecting an obstacle 
by projecting the image of a scene onto a plane different from the image plane (Page 
761 , RC, lines 23-25), by mapping a point in the image plane onto a point in a plane 
slanted (inclined) by an angel and determines the inclination (page 762, LC, equation 
1 ). The map of free space is the projection of all lines of sights onto a plane 
representing the floor up to the point where the respective line of sight stops. The 
inverse perspective mapping offers an efficient way of computing the map of free space 
for a broad range of applications of automatically detecting an obstacle. An obstacle is 
modeled as any visible surface patch whose 3D position has a non-vanishing Z 
component in an XYZ coordinate system with the XY-plane representing the floor (Z=0) 
(page 763, RC, lines 16-33, Figs 8, 9). It would have been obvious to one having 
ordinary skill in the art at the time the invention was made to use Storjohann teachings 
to modify the combined system of Onoguchi'440 and Onoguchi'483 by determines the 
inclinations of said lines on the images in order to efficiently computing the map of free 
space for a broad range of applications of automatically detecting an obstacle. 

As to claim 21/17 refer to claim 5/1 rejection. 
9. Claims 5/2, 5/3, 21/18, 21/19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Onoguchi Kazunori (Japanese Patent Publication 
1 1-102440) "referred to as Onoguchi'440 M , Onoguchi (U.S. Patent 5,694,483) "referred 
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to as Onoguchi' 483", and Applicant's Admitted Prior Art (page 3, line 4-page 4, Iine18) 
as applied to claims 2, 3, 18 and 19 above and further in view of K. Storjohann et al., 
"Visual Obstacle Detection For Automatically Guided Vehicles", IEEE International 
Conference on Robotics and Automation, p761-766, 1990. 

As to claims 5/2 or 5/3, Onoguchi'483 further discloses, wherein said feature 
extraction unit extracts a plurality of lines, as existing on the plane in the three- 
dimensional space and parallel to each other in the three-dimensional space, from the 
images [Fig. 10, shows the two lines of the road are spaced a distance W and run 
parallel to each other in the 3D space of plane P], and the vanishing points of said lines 
[Fig. 3, shows the cross point (vanishing point) of the lines is determined]. Neither 
Onoguchi'440 nor Onoguchi' 483 nor Applicant's Admitted Prior Art 

determines the inclinations of said lines on the images. 

Storjohann discloses an inverse perspective mapping for detecting an obstacle 
by projecting the image of a scene onto a plane different from the image plane (Page 
761 , RC, lines 23-25), by mapping a point in the image plane onto a point in a plane 
slanted (inclined) by an angel and determines the inclination (page 762, LC, equation 
1 ). The map of free space is the projection of all lines of sights onto a plane 
representing the floor up to the point where the respective line of sight stops. The 
inverse perspective mapping offers an efficient way of computing the map of free space 
for a broad range of applications of automatically detecting an obstacle. An obstacle is 
modeled as any visible surface patch whose 3D position has a non-vanishing Z 
component in an XYZ coordinate system with the XY-plane representing the floor (Z=0) 
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(page 763, RC, lines 16-33, Figs 8, 9). It would have been obvious to one having 
ordinary skill in the art at the time the invention was made to use Storjohann teachings 
to modify the combined system of Onoguchi'440, Onoguchi'483 and Applicant's 
Admitted Prior Art by determines the inclinations of said lines on the images in order to 
efficiently computing the map of free space for a broad range of applications of 
automatically detecting an obstacle. 

As to claims 21/18 or 21/19 refer to claim 5/2 or 5/3 rejections. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Samir A. Ahmed whose telephone number is 703-305- 
9870. The examiner can normally be reached on Mon-Fri 8:30am-6:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 703-308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




